METHOD OF FABRICATING PRINTED MATERIAL FOR STEREOSCOPIC 
VIEWING AND PRINTED MATERIAL FOR STEREOSCOPIC VIEWING 



Japanese Patent Application Nos. 2002-328834 and 2002-328869, filed on 
November 12, 2002, are hereby incorporated by reference in their entirety. 

BACKGROUND OF THE INVENTION 

The present invention relates to a method of fabricating printed material for 
stereoscopic viewing, together with printed material for stereoscopic viewing. 

A technique is known in the art of preparing an image for the left eye, taken by 
a camera that is equivalent to the left eye, and an image for the right eye, taken by a 
camera that is equivalent to the right eye, then combining these images by anaglyph 
processing or the like to obtain a stereoscopic image (printed material for stereoscopic 
viewing), as disclosed in Japanese Patent Laid-Open No. 2000-56411, by way of 
example. 

There are three physiological functions that enable a human being to experience 
the feeling of three dimensions: (1) binocular parallax (difference in the line-of-sight 
angles thereof) that creates an offset in the imaging of the retinas due to the spatial 
separation of the left and right eyes, (2) convergence, which is a function that directs 
the eyes inward, and (3) focal adjustment (focal distance) of adjusting the thickness of 
the lens in answer to distance to the object. The human being processes the three 
physiological functions of binocular parallax, convergence, and focal adjustment within 
the brain, to produce the stereoscopic feeling. 

The relationships between these three physiological functions are usually linked 
within the brain. If an error or inconsistency should arise in these relationships, 
therefore, the brain will try to force these stereoscopic linkages, which creates a 
situation in which the image feels unnatural or the viewer cannot recognize it as a 
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stereoscopic image. 

Incidentally, stereoscopic viewing in the prior art used only binocular parallax 
and convergence for representing a stereoscopic view. For that reason, the focus (focal 
distance) to any point within the surface of the stereoscopic image (printed material for 
5 stereoscopic viewing) remains substantially constant, so that offsets in binocular 
parallax or convergence occur at substantially all the locations in the stereoscopic 
image, making it impossible to implement a stereoscopic view that does not force the 
brain of a human being. 

1 o BRIEF SUMMARY OF THE INVENTION 

According to one aspect of the present invention, there is provided a method of 
fabricating printed material for stereoscopic viewing, the method comprising: 
creating a first image for the left eye for stereoscopic viewing; 
creating a first image for the right eye for stereoscopic viewing; 
15 creating a second image for the left eye by subjecting the first image for the left 

eye to correction processing for removing perspective of an image at a base surface in 
the first image for the left eye; 

creating a second image for the right eye by subjecting the first image for the 
right eye to correction processing for removing perspective of an image at the base 
20 surface in the first image for the right eye; and 

fabricating printed material for stereoscopic viewing based on the second image 
for the left eye and the second image for the right eye. 

According to another aspect of the present invention, there is provided a method 
of fabricating printed material for stereoscopic viewing, the method comprising: 
25 creating a first image for the left eye for stereoscopic viewing by photographing 

a subject and first to fourth marks for the left eye from a viewpoint position for the left 
eye, the first to fourth marks for the left eye forming a rectangle on a base surface; 
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creating a first image for the right eye for stereoscopic viewing by 
photographing the subject and first to fourth marks for the right eye from a viewpoint 
position for the right eye, the first to fourth marks for the right eye forming a rectangle 
on the base surface; 

5 creating a second image for the left eye from the first image for the left eye 5 by 

performing correction processing which moves the first to fourth marks for the left eye 
in the first image for the left eye to positions of vertices of a rectangle; 

creating a second image for the right eye from the first image for the right eye, 
by performing correction processing which moves the first to fourth marks for the right 
10 eye in the first image for the right eye to positions of vertices of a rectangle; and 

fabricating printed material for stereoscopic viewing based on the second image 
for the left eye and the second image for the right eye. 

According to a further aspect of the present invention, there is provided printed 
material for stereoscopic viewing which is fabricated by combining an image for the 
1 5 left eye and an image for the right eye, 

wherein each of the image for the left eye and the image for the right eye 
comprises an image of an object which is disposed on a base surface, and 

wherein the image of the object in the image for the left eye and the image of 
the object in the image for the right eye match at the base surface, and an offset between 
20 the image of the object in the image for the left eye and the image of the object in the 
image for the right eye becomes larger with increasing distance from the base surface. 

According to still another aspect of the present invention, there is provided a 
method of fabricating printed material for stereoscopic viewing, the method 
comprising: 

25 creating an image for the left eye by rendering a projection of each point of a 

graphic object onto a base surface which is not orthogonal to a line-of-sight direction, in 
a projection direction linking a viewpoint position for the left eye to each point of the 
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graphic object; 

creating an image for the right eye by rendering a projection of each point of the 
graphic object onto the base surface, in a projection direction linking a viewpoint 
position for the right eye to each point of the graphic object; and 
5 fabricating printed material for stereoscopic viewing, based on the image for the 

left eye and the image for the right eye. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
Fig. 1 is a flowchart of a first stereoscopic viewing method in accordance with 
1 0 an embodiment of the present invention; 

Fig. 2 is illustrative of the first stereoscopic viewing method in accordance with 
an embodiment of the present invention; 

Fig. 3 shows an example of an image IL1 for the left eye; 
Fig. 4 shows an example of an image IR1 for the right eye; 
1 5 Fig. 5 shows an example of an image IL2 for the left eye; 

Fig. 6 shows an example of an image IR2 for the right eye; 
Fig. 7 shows an example of a stereoscopic image (an anaglyph image); 
Figs. 8A, 8B, and 8C are illustrative of correction processing to remove 
perspective; 

20 Fig. 9 is illustrative of characteristics of a stereoscopic image that is obtained by 

an embodiment of the present invention; 

Fig. 10 is illustrative of a method of providing a plurality of base surfaces; 

Fig. 1 1 is a flowchart of the method of providing a plurality of base surfaces; 

Fig. 12 is further illustrative of the method of providing a plurality of base 
25 surfaces; 

Fig. 13 is illustrative of a second stereoscopic viewing method in accordance 
with an embodiment of the present invention; 
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Figs. 14A 5 14B ; and 14C are further illustrative of the second stereoscopic 
viewing method; 

Figs. 15A and 15B are illustrative of a prior-art method; 

Fig. 16 is illustrative of a method of setting a viewpoint position; 
5 Fig. 17 is illustrative of a method of using photographic images for fabricating 

printed material for stereoscopic viewing; 

Fig. 1 8 is further illustrative of the method of using photographic images for 
fabricating printed material for stereoscopic viewing; 

Figs. 19A and 19B are illustrative of a method of using CG images for 
10 fabricating printed material for stereoscopic viewing; 

Fig. 20 is illustrative of correction processing using texture mapping; 

Fig. 21 shows an example of a configuration of an image generation system; 

Fig. 22 is illustrative of a coordinate system; 

Fig. 23 is illustrative of the conversion from Gl to G4 to Gl' to G4' and from 
15 Gl'toG4'toFl toF4; 

Fig. 24 is illustrative of the derivation of conversion equations of the second 
stereoscopic viewing method; 

Figs. 25A and 25B are illustrative of conversions that form the second 
stereoscopic viewing method; 
20 Fig. 26 is further illustrative of conversions that form the second stereoscopic 

viewing method; and 

Fig. 27 is illustrative of an offset magnitude during the conversion of the first 
stereoscopic viewing method. 

25 DETAILED DESCRIPTION OF THE EMBODIMENT 

The embodiments of the present invention are described below with reference to 
the drawings. 
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Note that the embodiments described below do not limit the scope of the 
invention defined by the claims laid out herein. Similarly, the overall configuration of 
the embodiments below should not be taken as limiting the subject matter defined by 
the claims herein. 

5 This embodiment implements a stereoscopic view by either of two methods, as 

described below. 

1 . First stereoscopic viewing method 

A flowchart of a first stereoscopic viewing method in accordance with this 

1 0 embodiment is shown in Fig. 1 . 

First of all, the system creates (generates) a first image IL1 for the left eye and a 
first image IR1 for the right eye, for stereoscopic viewing (steps SI and S2). More 
specifically, the image IL1 for the left eye as seen from a viewpoint position VPL for 
the left eye and the image IR1 for the right eye as seen from a viewpoint position for 

1 5 the right eye VPR are created (generated). 

In this case, the viewpoint positions for the left eye and the right eye VPL and 
VPR are positions at which the left eye and the right eye of a viewer are assumed to be, 
as shown in Fig. 2. If, for example, images IL1 and IR1 for the left eye and the right eye 
are created by photographing with a camera (digital camera), the camera is positioned at 

20 these VPL and VPR positions to capture the images IL1 and IR1 for the left eye and the 
right eye. In such a case, two cameras disposed at VPL and VPR could capture pictures 
simultaneously, or the position of one camera could be changed for the captures. 

Alternatively, if the images IL1 and IR1 for the left eye and the right eye are 
generated by a system that generates computer graphics (CG) images and game images 

25 (real-time moving images), virtual cameras are disposed at these VPL and VPR 
positions to generate the images IL1 and IR1 for the left eye and the right eye. In other 
words, images are obtained as seen from VPL and VPR in the object space. 
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Examples of the image IL1 for the left eye and the image IR1 for the right eye 

as shown in Figs. 3 and 4. These are examples in which the images IL1 and IR1 have 

been created by photography with a camera (digital camera). Various objects 

(synonymously or in a narrow sense: substances, subjects, items, hereinafter the same) 

5 such as a tangerine, a box, a ballpoint pen, and a stapler are disposed on a base surface 

(a mounting surface on which the objects are placed). The image IL1 for the left eye is 

obtained by disposing a camera at the viewpoint position VPL for the left eye and 

capturing the image by directing the line of sight (direction) of the camera toward the 

object (the point of regard, or a representative point of the object). Similarly, the image 

10 IR1 for the right eye is obtained by disposing the camera at the viewpoint position for 

the right eye VPR and directing the line of sight of the camera toward the object. As 

shown in Figs. 3 and 4, there is an offset between the line-of-sight angles (viewing 

directions) of these images IL1 and IR1 for the left eye and the right eye, and a 

stereoscopic view is implemented by using the binocular parallax caused by that offset 

1 5 in the line-of-sight angles. 

Note that in this embodiment the surface (desk or table surface, or other 

horizontal surface) on which the printed material for stereoscopic viewing is placed 

during the stereoscopic viewing can be set to be a base surface. 

With CGs or a game, graphic objects (such as graphic objects that model a 
20 tangerine, a box, a ballpoint pen, and a stapler) are disposed on a base surface that has 

been set within an object space, and virtual cameras are positioned at VPL and VPR. 

Images similar to those of Figs. 3 and 4 can be generated by directing the line-of-sight 

(direction) of each virtual camera towards the graphic objects (point of regard, or 

representative point of the graphic objects) then generating images as seen from the 
25 virtual cameras. 

As shown in step S3 of Fig. 1, correction processing for removing the 

perspective of the images at a base surface BS is done on the first image IL1 for the left 
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eye obtained by step SI, to create (generate) a second image IL2 for the left eye. 
Similarly, as shown at step S4 3 correction processing for removing the perspective of 
the images at the base surface BS is done on the first image IR1 for the right eye 
obtained by step S2, to create (generate) a second image 1R2 for the right eye. 
5 Examples of the image IL2 for the left eye and the image IR2 for the right eye 

obtained by the correction processing are shown in Figs. 5 and 6. As shown by way of 
example in Figs. 3 and 4, a rectangle RTG (where "rectangle" has a general meaning 
that also comprises "square", which is the same hereinafter) that is drawn on the base 
surface BS is also in perspective. In contrast thereto, the perspective of the rectangle 

1 0 RTG has been removed in Figs. 5 and 6. 

In this case, the correction processing for removing perspective of this 
embodiment is processing for removing perspective (depth perception) from an image 
of the base surface BS itself, an image IM1 drawn on the base surface, and an image of 
portions of an object OB (graphic object) that are in contact with the base surface BS, 

15 as shown in Fig. 8 A. In other words, whereas the distance between vertices becomes 
smaller further away from the viewpoint in Bl of Fig. 8A, the distance between the 
vertices does not change further away from the viewpoint in B2 of Fig. 8A. The 
execution of such correction processing makes it possible to create (generate) an image 
as if seen from above, for the image of the base surface BS. Note that this correction 

20 processing need not necessarily remove the perspective completely; it could equally 
well remove enough of the perspective to avoid incongruity in the stereoscopic view. 

As shown at step S5 of Fig. 1, a stereoscopic image (image data) is created 
(generated), based on the second image .IL2 for the left eye and the second image IR2 
for the right eye. More specifically, anaglyph processing or the like based on IL2 and 

25 IR2 is done to create (generate) a stereoscopic image. 

A color printer (generally speaking: a printing machine) using a method such as 
an inkjet method or a laser printer method is then used to print this stereoscopic image 
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(photographic image or CG image) onto a print medium, to fabricate printed material 
for stereoscopic viewing. Note that the printed material for stereoscopic viewing could 
also be fabricated by making a master print by the color printer (print machine) then 
copying this printed material for stereoscopic viewing. This has the advantage of 
5 making it possible to fabricate large volumes of the printed material for stereoscopic 
viewing within a short period of time. 

Displaying the stereoscopic image on the display section of an image generation 
system makes it possible to generate game images (moving images) in real time. Note 
that in this case, stereoscopic images that have been obtained by anaglyph processing 

10 could be displayed directly on the display section and viewed by using spectacles 
(generally speaking: an appliance) that have been provided with color filters (red and 
blue). Alternatively, the images IL2 and IR2 for the left eye and the right eye could be 
displayed alternately on the display section with different frames and viewed by using 
spectacles that have been provided with liquid-crystal shutters or the like. 

15 An example of a stereoscopic image obtained by anaglyph processing based on 

the images IL2 and IR2 for the left eye and the right eye of Figs. 5 and 6 is shown in 
Fig. 7. 

This stereoscopic image shown in Fig. 7 is a composite of the image IL2 for the 
left eye (IL) and the image IR2 for the right eye (IR). The image IL2 for the left eye and 

20 the image IR2 for the right eye each comprise an image of the object OB disposed on 
the base surface BS. An image of the base surface BS is also comprised therein. 

As shown at Al in Fig. 9, the object image of the image IL2 for the left eye and 
the object image of the image IR2 for the right eye match at the position of the base 
surface BS (note that it is not always necessary that they should match completely). In 

25 other words, the print position (display position) of the object image of the image IL2 
for the left eye and the print position (display position) of the object image of the image 
IR2 for the right eye match at the base surface BS. 
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With increasing distance from the base surface BS, however, the offset between 
the object image of the image IL2 for the left eye and the object image for the image 
IR2 for the right eye grows larger, as shown at A2 in Fig. 9. More specifically, portions 
of the object OB that are positioned higher than the base surface BS have a larger offset 
between the print position (display position) for the image IL2 for the left eye and the 
print position (display position) for the image IR2 for the right eye. 

Printed material for stereoscopic viewing can be fabricated by printing the 
stereoscopic images shown in Figs. 7 and 9 on a print medium. A stereoscopic view can 
be implemented by viewing this printed material for stereoscopic viewing through 
spectacles provided with a red-color filter for the left eye and a blue-color filter for the 
right eye, by way of example. In addition, a game stereoscopic image can be generated 
by displaying the stereoscopic image of Figs. 7 and 9 on a display section. 

Conventionally, the printed material for stereoscopic viewing PM (or display 
screen of a display section, hereinafter the same) has been assumed to be disposed with 
the surface thereof parallel to a vertical surface, with the viewer looking at the printed 
material for stereoscopic viewing PM face-on, as shown in Fig. 8B. For that reason, 
images IL1 and IR1 for the left eye and the right eye such as those of Figs. 3 and 4 are 
subjected to anaglyph processing without modification, to create the printed material 
for stereoscopic viewing PM. Since some perspective does remain in the images of Figs. 
3 and 4, the printed material for stereoscopic viewing PM shown in Fig. 8B will appear 
to be a correct image if viewed face-on, but only from the perspective perception point 
of view. 

However, when the viewer looks at the printed material for stereoscopic 
viewing PM face-on, as shown in Fig. 8B, the focus (focal distance) to any point of the 
entire surface of the PM ends up the same. An inconsistency or error in the relationship 
between focal adjustment and binocular parallax or convergence will therefore be 
created in the brain of the human being. The brain will therefore be forced to create 
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stereoscopic linkages, which will seem unnatural and will make it impossible to 
recognize the stereoscopic effect. If the printed material for stereoscopic viewing PM 
created by the prior-art method is looked at while disposed on a desk that is parallel to 
the horizontal surface, there will be inconsistencies in the depth perception, giving an 
5 unnatural stereoscopic view. In other words, the rectangle RTG of Figs. 3 and 4 is a 
plan view with a height of zero, so that the rectangle RTG cannot be viewed 
stereoscopically. 

With this embodiment, it is assumed that the viewer looks at the printed material 
for stereoscopic viewing PM (display screen) while it is disposed on a desk (with the 

10 base surface BS parallel to a horizontal surface), as shown in Fig. 8C. In other words, 
this disposition is the default disposition of this method. When the printed material for 
stereoscopic viewing PM is to be disposed parallel to a horizontal surface in this 
manner, creating the printed material for stereoscopic viewing PM by subjecting the 
images of Figs. 3 and 4 to anaglyph processing without modification will create 

1 5 inconsistencies in the perspective. 

With this embodiment, correction processing is executed to remove the 
perspective of the image of the base surface, as described with reference to Figs. 5, 6, 
and 8A. The anaglyph processing is then performed based on the images of Figs. 5 and 
6, after the removal of the perspective of the base surface, to create the printed material 

20 for stereoscopic viewing PM, so that if the thus created printed material for stereoscopic 
viewing PM is disposed parallel to a horizontal surface as shown in Fig. 8C, the image 
of the base surface (the rectangle RTG) has a suitable perspective. If the printed 
material for stereoscopic viewing PM is disposed as shown in Fig. 8C, the focal 
distance to each point of the surface thereof will not be the same, but will differ. For 

25 that reason, the focal adjustment will be similar to that in the real world. The offsets in 
the relationships between the focal adjustment and binocular parallax or convergence 
are also therefore reduced, making it possible to implement a more natural stereoscopic 
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view that has a feeling of reality. 

Note that there is a possibility that an offset in depth perception will occur in 
stereoscopic viewing method of this embodiment if the height of the object is too high. 
In such a case, two base surfaces BS1 and BS2 (generally speaking: a plurality of base 
surfaces) could be provided, as shown in Fig. 10. 

In this case, the base surface BS1 is a surface that is parallel to a horizontal 
surface, by way of example. The base surface BS2, on the other hand, is a surface that 
subtends a predetermined angle (such as a right angle) with the base surface BS1. The 
base surfaces BS1 and BS2 are linked by a boundary BD. 

The object OB (graphic object) is disposed on the base surface BS1 and also on 
the nearer side of the base surface BS2 (the side with VPL and VPR). The processing is 
performed as shown in Fig. 1 1 instead of that of Fig. 1 . 

Steps Sll and S12 of Fig. 11 are similar to steps SI and S2 of Fig. 1. At step 
SI 3, correction processing to remove the perspective at the base surface BS1 is 
performed for an area corresponding to the base surface BS1 for the image IL1 for the 
left eye (a first area of IL1 on the BS1 side with reference to the boundary BD). 
Correction processing for removing perspective at the base surface BS2 is then 
performed for an area corresponding to the base surface BS2 for IL1 (a second area of 
IL1 on the BS2 side with reference to the boundary BD). The image IL2 for the left eye, 
which is an image connecting the images generated by this correction processing, is 
then created (generated). 

In step SI 4, correction processing to remove the perspective at the base surface 
BS1 is performed for an area corresponding to the base surface BS1 for the image IR1 
for the right eye (a first area of IR1 on the BS1 side with reference to the boundary BD). 
Correction processing for removing perspective at the base surface BS2 is then 
performed for an area corresponding to the base surface BS2 for IR1 (a second area of 
IR1 on the BS2 side with reference to the boundary BD). The image IR2 for the right 
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eye, which is an image connecting the images generated by this correction processing, 
is then created (generated). 

In the final step SI 5, processing such as anaglyph processing is used to create 
(generate) a stereoscopic image, based on IL2 and IR2. The thus-obtained stereoscopic 
5 image could be printed on a print medium to produce printed material for stereoscopic 
viewing, or a game image that is a real-time moving image could be generated by 
displaying the stereoscopic image on a display section. 

This process makes it possible to implement a more natural stereoscopic view 
that has a feeling of reality, even when OB is an object that stands some height above 
10 the base surface BS1, as shown in Fig. 12. In other words, it is possible to implement a 
stereoscopic view in an area in the vicinity of the feet of the object OB (the first area on 
the lower side of the boundary BS with a natural depth perception or focal adjustment, 
by processing using the base surface BS1. Similarly, it is possible to implement a 
stereoscopic view in the other areas (the second area on the upper side of the boundary 
15 BS) with a natural depth perception, by processing using the base surface BS2. 

Note that the number of base surfaces is not limited to two, and thus three or 
more base surfaces (a plurality of linked base surfaces) could be used. 

2. Second stereoscopic viewing method 
20 A flowchart of a second stereoscopic viewing method in accordance with this 

embodiment is shown in Fig. 13. In contrast to the previously described method of Fig. 

1, which is the optimal method for using images captured by a camera to create printed 

material for stereoscopic viewing, the method of Fig. 1 3 is the optimal method for using 

CG images to create printed material for stereoscopic viewing. 
25 First of all, the system projects each point of the graphic object OB onto the 

base surface BS (either BS1 or BS2 in the setup of Fig. 10) in a projection direction 

linking the viewpoint position VPL for the left eye to each point of the OB, to render it 
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onto the base surface BS and create (generate) the image IL for the left eye (step S21). 

It then projects each point of the graphic object OB onto the base surface BS 
(either BS1 or BS2 in the setup of Fig. 10) in a projection direction linking the 
viewpoint position VPR for the right eye to each point of the OB 5 to render it onto the 
5 base surface BS and create (generate) the image IR for the right eye (step S22). Note 
that the base surface BS is a surface that is not orthogonal to the line-of-sight direction 
(the direction linking the viewpoint position and a point of regard), by way of example. 
In other words, the base surface BS is a different surface from a perspective projection 
screen that is always orthogonal to the line-of-sight direction. 

10 The processing of steps S21 and S22 projects a virtual beam of light from VPL 

(or VPR) toward the graphic object OB, in such a manner that the light is used to burn 
an image of the OB onto virtual paper that is the base surface BS (BS1 or BS2), to 
render it onto the virtual paper. This renders images (having properties such as color) of 
points PI, P2, P3, and P4 of the graphic object OB into projected points PI', P2', P3', 

15 and P4' on the base surface BS. Note that images of points P5 and P6 on the base 
surface BS are rendered without change at the positions of those points P5 and P6. It is 
possible to create an image IL for the left eye and an image IR for the right eye, similar 
to IL2 and IR2 of Figs. 5 and 6, by rendering the entire surface of the base surface BS 
(virtual paper) by a raster scan, as shown by way of example in Fig. 14B. In other 

20 words, it is possible to create images IL and IR for the left eye and the right eye from 
which the perspective of the image of the base surface has been removed. 

The system then performs anaglyph processing or the like, based on these 
images IL and IR for the left eye and the right eye, to create (generate) the stereoscopic 
image (step S23). This makes it possible to obtain a stereoscopic image such as that 

25 shown in Fig. 7. The thus-obtained stereoscopic image could be printed on a print 
medium to produce printed material for stereoscopic viewing, or a game image could be 
generated by displaying the stereoscopic image on a display section. 
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This makes it possible to implement a more natural stereoscopic view that has a 
feeling of reality, by disposing the printed material for stereoscopic viewing PM (or the 
display screen) so that it is parallel to a horizontal surface (base surface), as shown by 
way of example in Fig. 14C. 

The graphic object OB is projected onto a perspective projection screen SCR (a 
surface that is orthogonal to the line-of-sight direction) to create the image for the left 
eye and the image for the right eye, as shown by way of example in Fig. 15 A. The thus- 
obtained image for the left eye and image for the right eye are combined to create the 
printed material for stereoscopic viewing PM. The viewer looks at the printed material 
for stereoscopic viewing PM face-on, as shown in Fig. 15B. 

With this method of Fig. 15A, the points P2 and P3 of the graphic object OB are 
projected onto points P2" and P3" on the projection projection screen SCR. Since the 
printed material for stereoscopic viewing PM is viewed face-on, as shown in Fig. 15B, 
the difference L2 in focal distance between P2" and P3" is zero. In other words, 
although the difference LI in focal distance between the actual points P2 and P3 is not 
zero, L2 is zero and thus the focal adjustment differs from that in reality. This therefore 
creates an inconsistency in the relationship between focal adjustment and binocular 
parallax, creating confusion in the human brain which results in a stereoscopic view 
that has an incongruous feel. 

In contrast thereto, since the printed material for stereoscopic viewing PM 
(display screen) of this embodiment is seen while placed on a desk as shown in Fig. 
14C, the difference L2 in focal distance between the points P2 f and P3' is not zero, in a 
similar manner to the difference LI in focal distance between the actual points PI and 
P2, as shown in Fig. 14A. Since the nearer portion (point P2) is seen to be nearer and 
the farther portion (point P3) is seen to be farther away, therefore, there is no 
inconsistency in the relationship between focal adjustment and binocular parallax and 
thus no confusion is created in the human brain, making it possible to implement a 
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more natural stereoscopic view. 

In other words, since the method of this embodiment ensures that the printed 
material for stereoscopic viewing PM is placed on a desk and is seen at an angle, the 
surface of the desk and the base surface BS (zero surface) on which the graphic object 
5 OB is placed as the subject of the stereoscopic view are in the same plane, enabling a 
natural, realistic stereoscopic view. Since it suffices to represent an appearance in which 
the graphic object OB floats a few centimeters above the base surface BS (zero surface), 
there is substantially no inconsistency in the depthwise direction. Moreover, since the 
base surface BS is the surface of the desk, it appears that the stereoscopic object is 

10 disposed on that desk in reality, increasing the realness of the object. In other words, 
since the base surface is ambiguous with the prior-art method of Figs. 15A and 15B, the 
realness of the object can only make it seem like a phantom. 

Note that the method of Fig. 1 3 can also create (generate) a stereoscopic image 
in which a plurality of base surfaces are set, as described previously with respect to Fig. 

15 10. In such a case, steps S21 and S22 of Fig. 13 ensure that the points that are projected 
onto the base surface BS1 could be rendered onto the base surface BS1 and the points 
projected onto the base surface BS2 could be rendered onto that base surface BS2. 

3 . Anaglyph processing 

20 The anaglyph processing of step S5 of Fig. 1, step SI 5 of Fig. 11, and step S23 

of Fig. 13 will now be described simply. 

Anaglyph processing prints an image for the left eye and an image for the right 
eye in different colors on one sheet of print medium, to create a printed material for 
stereoscopic viewing. This printed material for stereoscopic viewing is viewed through 

25 color filters that differ for the left and right eyes (such as red for the left eye and blue 
for the right eye). During this time, the stereoscopic view is implemented by ensuring 
that the left eye sees only the image for the left eye and the right eye sees only the 
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image for the right eye. 

With monochrome anaglyph processing, for example, the image for the left eye 
(IL2 or IL) is converted to grayscale. The image data after the conversion is copied into 
the R channel of the anaglyph image (RGB). The image for the right eye (IR2 or IR) is 
5 then converted to grayscale. The image data after the conversion is copied into the G 
channel and the B channel of the anaglyph image (RGB). This creates a monochrome 
anaglyph image. Note that the image for the right eye could equally well be copied into 
the B channel alone. 

With color anaglyph processing, the R channel of the image for the left eye (IL2 

10 or IL) is copied into the R channel of the anaglyph image (RGB). Similarly, the G 
channel of the image for the right eye (IR2 or IR) is copied into the G channel of the 
anaglyph image (RGB). The B channel of the image for the right eye is copied into the 
B channel of the anaglyph image (RGB). This makes it possible to create a color 
(pseudo-color) anaglyph image. 

15 Note that the method of implementing the stereoscopic view (step S5 of Fig. 1, 

step S15 of Fig. 11, or step S23 of Fig. 13) is not limited to anaglyph processing, 
providing at least an image for the left eye (IL2 or IL) and an image for the right eye 
(IR2 or IR) are used. 

For instance, a special lens called a lenticular lens could be used to ensure that 

20 only the image for the left eye enters the left eye and only the image for the right eye 
enters the right eye, to implement the stereoscopic view. 

It is also possible to dispose polarizing plates in front of the image for the left 
eye and the image for the right eye, where the polarizing plate placed before the image 
for the left eye has a different polarization direction than the polarizing plate placed 

25 before the image for the right eye. The stereoscopic view could be implemented by the 
viewer wearing spectacles in which polarizing plates having corresponding polarization 
directions are inserted into the lens portions thereof. 
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The image for the left eye and the image for the right eye are displayed 
alternately at each frame, by way of example. The stereoscopic view could be 
implemented by the viewer wearing spectacles in which a shutter (such as a liquid- 
crystal shutter) for the left eye opens in synchronization with the display of the image 
5 for the left eye and a shutter for the right eye opens in synchronization with the display 
of the image for the right eye. 

4. Setting of viewpoint positions 

The description now turns to a method of setting the viewpoint positions. 
10 It is desirable that the viewpoint positions for the left eye and the right eye VPL 

and VPR shown in Figs. 2 and 1 0 are disposed at the assumed positions of the left and 

right eyes of the viewer when the viewer is actually viewing the printed material for 

stereoscopic viewing or the display screen for the stereoscopic view. In the example 

shown in Figs. 2 and 10, the viewpoint positions for the left eye and the right eye VPL 
1 5 and VPR are set on the basis of a distance DVB (such as 40 cm) between the object OB 

(graphic object, subject) and the viewer, a line-of-sight angle 0 (line-of-sight direction 

SL), and a distance between the eyes DLR (such as 7 cm). 

It should be noted, however, that if the display is reduced or enlarged, the 

positions of VPL and VPR are moved in accordance with the reduction ratio or 
20 enlargement ratio. In such a case, it is desirable that the viewpoint positions are moved 

by a method such as that shown in Fig. 16. 

If the distance DVB between the object OB (subject, graphic object) and the 

viewpoint position (a central point CP between VPL and VPR) becomes longer, by way 

of example, the distance DLR between the viewpoint position VPL for the left eye and 
25 the viewpoint position for the right eye VPR is increased in accordance with the change 

in this length (ratio). In other words, DLR is lengthened in proportion to the change in 

the length of DVB. 
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If the distance DVB between the object OB (subject, graphic object) and the 
viewpoint position (the central point CP between VPL and VPR) is changed, the 
viewpoint position (the central point CP, as well as VPL and VPR) is moved so as to 
move along a line LN (line-of-sight direction) that subtends the predetermined angle 0 
5 with respect to the base surface BS. 

Since this ensures that the distance DVB and the distance DLR change at a 
uniform ratio, even when VPL and VPR are moved, it makes it possible to prevent a 
situation in which the stereoscopic perception fails. This makes it possible to implement 
a reduced or enlarged display while maintaining a suitable stereoscopic perception. 

10 

5. Use of photographic images to create printed material for stereoscopic viewing 

The description now turns to details of a method of using photographic images 
to create (fabricate) printed material for stereoscopic viewing. The first method 
described with reference to Fig. 1 is applied in this case. 

15 When using photographic images, it is necessary to ensure that the environment 

at capture is reproduced unchanged. The cameras (such as digital cameras) used for the 
photography are therefore disposed in a layout that is close to the positional relationship 
when the viewer looks at the result. For example, the cameras for the photography are 
disposed on the assumption that the printed material for stereoscopic viewing or the like 

20 is placed on a standard desk and the viewer is looking at it while sitting on a chair. 

5.1 Case with one base surface 

If there is one base surface as shown in Fig. 2, the distance between the eyes 
DLR (approximately 7 cm), the distance DVB between the viewpoint and the subject 
25 OB, the line-of-sight angle 9, and a longitudinal dimension Dl and a lateral dimension 
D2 (print region) of the print paper are set. 

Cameras are then disposed at positions where the left and right eyes of the 
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viewer are assumed to be. A sheet of paper on which marks MK1 to MK4 (first to 
fourth marks) have been drawn, to act as markers of the print region (Dl and D2), is 
positioned. These marks MK1 to MK4 form vertices of a rectangle on the base surface 
BS (where "rectangle" has a general meaning that also comprises "square"). 
5 The subject OB to be photographed is then mounted on the paper. During this 

time, the subject OB to be photographed is placed in such a manner that the OB does 
not protrude outside of the rectangle (print region) configured of the marks MK1 to 
MK4, as seen from the camera positions. The cameras that have been set at the 
positions that are assumed to be the positions of the left and right eyes are then used to 
10 capture images that include the subject OB to be photographed and the marks MK1 to 
MK4, to create the images IL1 and IR1 for the left eye and the right eye shown in Figs. 
3 and 4. 

The captured images IL1 and IR1 for the left eye and the right eye are then 
fetched into an image generation system (personal computer or information processing 

15 device) and displayed on screen. The marks MK1 to MK4 of the paper are then located 
from the displayed images. 

As shown in Fig. 17, the marks MK1 to MK4 are moved to positions of vertices 
VX1 to VX4 of a rectangle (where "rectangle" has a general meaning that also 
comprises "square") having an aspect ratio of Dl to D2, to perform distortion correction 

20 processing of the images. This correction processing is done for each of the images IL1 
and IR1 for the left eye and the right eye to create the images IL2 and IR2 for the left 
eye and the right eye such as those shown in Figs. 5 and 6. 

The unwanted portions outside the print region are trimmed. Software for 
anaglyph processing is then used to create a stereoscopic image (anaglyph image) such 

25 as that shown in Fig. 7 from the images IL2 and IR2 for the left eye and the right eye. 
The thus-obtained stereoscopic image is printed on paper in a print region of 
dimensions Dl and D2 5 to complete the printed material for stereoscopic viewing. 
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5.2 Case with two base surfaces 

If there are two base surfaces as shown in Fig. 10, the distance between the eyes 
DLR (approximately 7 cm), the distance DVB between the viewpoint and the subject 
5 OB, the line-of-sight angle 0, and a longitudinal dimension Dl, a lateral dimension D2, 
and a height dimension D3 (print region) of the print paper are set. 

Cameras are then disposed at positions where the left and right eyes of the 
viewer are assumed to be. A first sheet of paper (the base surface BS1) on which marks 
MK1 to MK4 (first to fourth marks) have been drawn, to act as markers of the print 
10 region (Dl and D2), is positioned. These marks MK1 to MK4 form vertices of a 
rectangle on the base surface BS1 . 

A second sheet of paper (the base surface BS2) on which marks MK5 to MK8 
(fifth to eighth marks) have been drawn, to act as markers of the print region (D2 and 
D3) is placed behind the first sheet of paper and perpendicular thereto. These marks 
1 5 MK5 to MK8 form vertices of a rectangle on the base surface BS2. 

The subject OB to be photographed is then mounted on the first sheet of paper. 
During this time, the subject OB to be photographed is placed in such a manner that the 
OB does not protrude outside of the rectangle configured of the marks MK1 to MK4 or 
the rectangle (print region) configured of the marks MK5 to MK8, as seen from the 
20 camera positions. The cameras that have been set at the positions that are assumed to be 
the positions of the left and right eyes are then used to capture images that include the 
subject OB to be photographed and the marks MK1 to MK4 and MK5 to MK8, to 
create the images (photographs) IL1 and IR1 for the left eye and the right eye. 

The captured images IL1 and IR1 for the left eye and the right eye are then 
25 fetched into an image generation system (personal computer) and displayed on screen. 
The marks MK1 to MK4 and MK5 to MK8 of the paper are then located from the 
displayed images. Note that the marks MK3 and MK4 and the marks MK6 and MK5 
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could be the same. 

As shown in Fig. 18, the marks MK1 to MK4 are moved to positions of vertices 
VX1 to VX4 of a rectangle having an aspect ratio of Dl to D2, to perform distortion 
correction processing of the images. Similarly, the marks MK5 to MK8 are moved to 
5 positions of vertices VX5 to VX8 of a rectangle having an aspect ratio of D3 to D2, to 
perform distortion correction processing of the images. The thus-obtained two images 
are linked. This correction processing is done for each of the images IL1 and IR1 for 
the left eye and the right eye to create the images IL2 and IR2 for the left eye and the 
right eye. 

10 The unwanted portions outside the print region are trimmed. Software for 

anaglyph processing is then used to create a stereoscopic image (anaglyph image) from 
the images IL2 and IR2 for the left eye and the right eye. The thus-obtained 
stereoscopic image is printed on paper in a print region of dimensions Dl, D2, and D3, 
to complete the printed material for stereoscopic viewing. 

15 

6. Use of CG images to create printed material for stereoscopic viewing 

The description now turns to details of a method of using computer graphics 
(CG) images to create (fabricate) printed material for stereoscopic viewing. The second 
method described with reference to Fig. 13 is applied in this case. Note, however, that it 
20 is also possible to implement this with the first method of Fig. 1 . 

First of all, virtual cameras (viewpoints) are disposed in the object space with a 
layout that is close to the positional relationship when the viewer looks at the result. For 
example, the virtual cameras are disposed on the assumption that the printed material 
for stereoscopic viewing or the like is placed on a standard desk and the viewer is 
25 looking at it while sitting on a chair. 

The distance between the eyes DLR (approximately 7 cm), the distance DVB 
between the viewpoint and the graphic object OB, the line-of-sight angle 0, and a 
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longitudinal dimension Dl and a lateral dimension D2 (print region) of the print paper 
are set, as shown in Fig. 2. 

The virtual cameras are then disposed at positions where the left and right eyes 
of the viewer are assumed to be. The graphic object is disposed on the virtual paper (a 
5 virtual paper graphic object). 

A virtual beam of light is emitted from the viewpoint position VPL for the left 
eye towards the graphic object OB, and this light is used to render an image of the OB 
as if exposed on the virtual paper. This creates the image IL for the left eye. This 
processing is similar to the processing for projecting an image as seen by the eyes by a 
1 0 projector onto the virtual paper on the desk. 

A virtual beam of light is emitted from the viewpoint position VPR for the right 
eye towards the graphic object OB, and this light is used to render an image of the OB 
as if exposed on the virtual paper. This creates the image IR for the right eye. 

Anaglyph processing software is then used to create a stereoscopic image 
1 5 (anaglyph image) from the images IL and IR for the left eye and the right eye. The thus- 
obtained stereoscopic image is printed on paper in a print region of dimensions Dl and 
D2, to complete the printed material for stereoscopic viewing. 

Note that printed material for stereoscopic viewing can also be created by 
providing a plurality of base surfaces such as is shown in Fig. 10, using CG images. 
20 The graphic object OB projected onto the base surface BS could either be such 

that all of the graphic object is disposed on the base surface BS, as shown in Fig. 2, or 
part thereof is disposed in the further side of the base surface BS, as shown in Fig. 19A. 
Alternatively, the entire graphic object could be disposed in the further side of the base 
surface BS, as shown in Fig. 19B. 
25 In the example shown in Fig. 19A, points PI, P2, and P3 in the further side of 

the base surface BS are projected onto points Pl\ P2\ and P3' that are closer. This 
makes it possible to represent a feature such as a hole in the graphic object OB. It is 
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also possible to represent the appearance of the graphic object OB sinking into the 
virtual paper at a position CI in Fig. 19A. 

In the example shown in Fig. 19B too, points PI, P2, and P3 in the interior of 
the base surface BS are projected onto points PI', P2', and P3' that are closer. This 
5 makes it possible to represent graphic objects such as fish submerged below a water 
surface. Note that, when representing a semi-transparent thing such as a water surface, 
it is desirable to dispose a semi-transparent graphic object at the position of the base 
surface BS and perform alpha blending on this semi-transparent graphic object and the 
graphic object OB (points PI', P2\ and P3')- 
10 The method of this embodiment of using CG images in this way makes it 

possible to create the ideal printed material for stereoscopic viewing for attachment to a 
game manual. 

For example, a technical problem with a prior-art game manual that can only 
contain two-dimensional maps is that it is difficult for the player to grasp the contours 

1 5 or the like of such a map. 

In contrast thereto, use of the method of this embodiment makes it possible to 
attach a map that can be viewed stereoscopically, as printed matter in the game manual. 
For example, to enable the use of shape data of a map as game data, it is possible to use 
that game data to create a printed map for stereoscopic viewing, in a simple manner. 

20 The method of this embodiment also makes it possible to provided printed material for 
stereoscopic viewing that is best for clearly exhibiting the optimal stereoscopic 
perception when placed on a desk or the like for viewing. It is therefore possible to 
provide the optimal printed material for stereoscopic viewing for attachment to the 
game manual, which is easy and convenient for the player to use. 

25 Note that printed material for stereoscopic viewing that shows appropriate 

vehicles, tanks, and aircraft could also be appended to the game manual for a game that 
includes vehicles, tanks, and aircraft. Alternatively, application of the method of this 
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embodiment to a monster card game makes it possible to implement a card game in 
which monsters on the cards appear to jump out in a stereoscopic manner. Since a card 
game in particular is enjoyed by placing the cards on a horizontal surface such as a desk 
or table, the method of this embodiment is ideal for enabling the most effective 
5 stereoscopic view when the cards are on a horizontal surface (base surface). 

7. Generation of Game Images 

The description now turns to a method of generating game images that are real- 
time moving images. In this case, the first method described with reference to Fig. 1 is 
10 applied. It should be noted that it is also possible to implement it with the second 
method of Fig. 13. 

First of all, virtual cameras (viewpoints) are disposed in the object space with a 
layout that is close to the positional relationship when the player looks at the result. For 
example, the virtual cameras are disposed on the assumption that the printed material 
15 for stereoscopic viewing or the like is placed on a standard desk and the player is 
looking at it while sitting on a chair. 

The distance between the eyes DLR (approximately 7 cm), the distance DVB 
between the viewpoint and the graphic object OB, the line-of-sight angle 9, and a 
longitudinal dimension Dl and a lateral dimension D2 (display screen size) of the 
20 display screen are set, as shown in Fig. 2. 

The virtual cameras are then disposed at the viewpoint positions for the left eye 
and the right eye VPL and VPR where the left and right eyes of the player are assumed 
to be. The object OB that is the subject of the virtual cameras is disposed within the 
object space. These virtual cameras are basically pointed in the direction of the graphic 
25 object (graphic object of regard) within the object space, from the viewpoint positions 
for the left eye and the right eye VPL and VPR. 

Images for the left eye and the right eye IL1 and IR1, as seen from the virtual 
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cameras disposed at the viewpoint positions for the left eye and the right eye VPL and 
VPR 5 are then generated. The thus-generated images IL1 and IR1 for the left eye and 
the right eye are written to a texture area (texture space) in VRAM and those images are 
set on a texture image TEX 5 as shown in Fig. 20. 
5 The texture image TEX with perspective (see Figs. 3 and 4) is mapped onto a 

polygon PLG (primitive surface) of a rectangle (where "rectangle" has a general 
meaning that also comprises "square") of dimensions Dl and D2. More specifically, 
texture coordinates (TX1, TY1), (TX2, TY2) ? (TX3, TY3), and (TX4, TY4) of the 
texture image TEX are correlated with vertices VX1 VX2, VX3, and VX4 of the 
10 polygon PLG, to map the texture image TEX onto the polygon PLG. This makes it 
possible to generate an image of the base surface with perspective removed, as shown 
in Figs. 6 and 7. This texture mapping processing is done for each of the images IL1 
and IR1 for the left eye and the right eye to generate images IL2 and IR2 for the left eye 
and the right eye. 

15 Anaglyph processing is then used to combine the thus-obtained images IL2 and 

IR2 for the left eye and the right eye into a single stereoscopic image. The thus- 
combined stereoscopic image is output to the display section. 

Note that if the stereoscopic view is implemented by using liquid-crystal 
shutters or the like, the generated images IL2 and IR2 for the left eye and the right eye 

20 could be output alternately in different frames to the display section. 

8. Image Generation System 

A functional block diagram of the image generation system of this embodiment 
is shown in Fig. 21 . Note that the image generation system of this embodiment does not 
25 necessarily comprise all of the structural elements (components) shown in Fig. 21, and 
some of them could be omitted. 

This image generation system of Fig. 21 can be used as a system for generating 



26 



game images (real-time moving images). In addition, it is also possible to create 
stereoscopic images from CG images (static images), and use it as an image generation 
system (CG tools) for creating the printed material for stereoscopic viewing. It is also 
possible to use it as an image generation system for fetching photographic images 
5 captured by cameras, then creating a stereoscopic image from those photographic 
images, to create printed material for stereoscopic viewing. 

An operating section 160 is designed to enable an player (operator) to input 
operating data, where the functions thereof can be implemented by hardware such as 
levers, buttons, a steering wheel, a shift lever, an accelerator pedal, a brake pedal, a 

1 0 microphone, sensors, a touch-panel, or a casing. 

A storage section 1 70 acts as a work area for components such as the processing 
section 100 and the communications section 196, where the functions thereof can be 
implemented by hardware such as RAM. 

An information storage medium 180 (a storage medium that can be read by a 

15 computer) is designed to store information such as programs and data, and its functions 
could be implemented by hardware such as an optical disk (CD or DVD), a magneto- 
optical (MO) disk, a magnetic disk, a hard disk, magnetic tape, or ROM. The 
processing section 100 performs the various types of processing in accordance with this 
embodiment, based on the information stored in this information storage medium 180. 

20 In other words, within the information storage medium 180 is stored (recorded, 
accommodated) a program (a program for causing the implementation of processing on 
a computer) that causes the computer to function as the various components of this 
embodiment. 

A display section 190 is designed to output images generated by this 
25 embodiment of the present invention, and the functions thereof can be implemented by 
hardware such as a CRT, LCD panel, touch-panel, or head-mounted display (HMD). 

A sound output section 192 is designed to output sounds created by this 
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embodiment of the present invention, and the functions thereof can be implemented by 
hardware such as a speaker or headphones. 

A portable information storage device 194 stores data such as a player's personal 
data and saved game data, and various devices such as a memory card or a portable 
5 game machine could be conceived as this portable information storage device 194. 

The communications section 196 provides various types of control for 
communicating with an external device (such as a host device or another image 
generation system), and the functions thereof can be implemented by hardware such as 
various types of processor or a communications ASIC, or by a program. Use of this 
10 communications section 196 makes it possible to fetch the photographic images 
captured by cameras into the image generation system, and output the created 
stereoscopic images to a printer. 

Note that a program or data for causing the implementation of the various 
processes of the present invention (this embodiment) on a computer could be 
15 distributed to the information storage medium 180 (the storage section 170) from an 
information storage medium possessed by a host device (server), through a network and 
the communications section 196. Such use of an information storage medium on the 
host device (server) is also comprised within the scope of the present invention. 

The processing section 1 00 (processor) performs various types of processing, 
20 such as game processing, image generation processing, or sound generation processing, 
based on manipulation data or a program from the operating section 160 or a program. 
In such a case, the processing section 100 uses a main storage section 172 within the 
storage section 170 as a work area to perform various types of processing. The 
functions of this processing section 100 could be implemented by hardware such as 
25 various different processors (such as a CPU or DSP) or an ASIC (gate array), or by a 
program (game program). 

The processing section 100 comprises a game processing section 110, a first 
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image generation section 120, a second image generation section 122, stereoscopic 
image generation section 126, and a sound generation section 130. 

In this case, the game processing section 1 1 0 performs various types of game 
processing based on manipulation data from the operating section 1 60 (game controller). 
5 This game processing includes processing such as that for starting the game, based on 
game start conditions; processing to move the game forward; processing for disposing 
the graphic objects (display objects) that appear in the game; processing for obtaining 
movement information (position, speed, and acceleration) and action information 
(motion information) of the graphic objects; processing for displaying the graphic 

10 objects; processing for calculating game results; or processing for ending the game 
when a game end condition has been satisfied. 

The first image generation section 120 performs processing to generate a first 
image for the left eye that is an image as seen from a viewpoint position for the left eye 
(a virtual camera for the left eye) within the object space. It also performs processing to 

15 generate a first image for the right eye that is an image as seen from a viewpoint 
position for the right eye (a virtual camera for the right eye) within the object space. In 
this case, the first image for the left eye and the first image for the right eye are 
stereoscopic images, such as images exhibiting binocular parallax. More specifically, a 
virtual camera is disposed at the viewpoint position for the left eye and the line-of- sight 

20 direction of that virtual camera is pointed towards the of the graphic object (point of 
regard) to generate a first image for the left eye. Similarly, a virtual camera is disposed 
at the viewpoint position for the right eye and the line-of-sight direction of that virtual 
camera is pointed towards the graphic object (point of regard) to generate a first image 
for the right eye. 

25 Note that images as seen from a virtual camera can be generated as described 

below. In other words, geometry processing such as coordinate conversion, clipping 
processing, transparency conversion, or light source processing is first performed, then 
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drawing data (such as position coordinates of the vertices of primitive surfaces, texture 
coordinates, color data, normal vectors, or a values) are created based on the results of 
that processing. Objects (comprising one or a plurality of primitive surfaces) after the 
transparency conversion (after geometry processing) are drawn into a drawing buffer 
5 1 74 (a buffer that enables the storage of image information in pixel units, such as a 
frame buffer or work buffer), based on this drawing data (primitive surface data). This 
causes the generation of images as seen from virtual cameras within the object space. 

The second image generation section 122 performs correction processing on the 
first image for the left eye to remove the perspective of the image at the base surface, to 

10 generate the second image for the left eye. It also performs correction processing on the 
first image for the right eye to remove the perspective of the image at the base surface, 
to generate the second image for the right eye (see Figs. 1 and 8A). 

Correction processing in this case is implemented by a texture mapping section 
124 performing the texture mapping described with reference to Fig. 20. More 

15 specifically, the first image for the left eye and the first image for the right eye 
generated by the first image generation section 120 are stored as texture images in a 
texture storage section 176. The texture mapping section 124 generates the second 
image for the left eye by mapping the thus-stored first image for the left eye onto a 
rectangular polygon. Similarly, it generates the second image for the right eye by 

20 mapping the thus-stored first image for the right eye onto the rectangular polygon. 

The second image generation section 122 could also use the method described 
with reference to Fig. 1 3 to generate the image for the left eye and the image for the 
right eye. In other words, the second image generation section 122 generates the image 
for the left eye by rendering a projection of each point of the graphic object onto the 

25 base surface, in the projection direction linking the viewpoint position for the left eye to 
each point of the graphic object. It also generates the image for the right eye by 
rendering a projection of each point of the graphic object onto the base surface, in the 



30 



projection direction linking the viewpoint position for the right eye to each point of the 
graphic object. 

The stereoscopic image generation section 126 performs processing to generate 
the stereoscopic image, based on the second image for the left eye (image for the left 
5 eye) and the second image for the right eye (image for the right eye). For example, it 
combines the second image for the left eye (image for the left eye) and the second 
image for the right eye (image for the right eye) by anaglyph processing to generate the 
stereoscopic image, and outputs it to the display section 190. In such a case, the player 
wears spectacles provided with a red-colored filter and a blue-colored filter for the left 
10 eye and the right eye, to play the game. 

Alternatively, the stereoscopic image generation section 126 could implement a 
stereoscopic view by processing that outputs the second image for the left eye (image 
for the left eye) and the second image for the right eye (image for the right eye) in 
different frames to the display section 190. In such a case, the player wears spectacles 
15 provided with shutters that open and close in synchronization with the frames, to play 
the game. 

The sound generation section 130 performs sound processing based on the 
various processing results obtained by the processing section 100, to generate games 
sounds such as background music, sound effects, and voices, and outputs them to the 
20 sound output section 192. 

Note that the image generation system of this embodiment could be applied to a 

system with a dedicated single-player mode, which enables one player to play the game, 

or it could also be applied to a system provided with a multi-player mode, which 

enables a plurality of players to play. 
25 If a plurality of players are playing, the game images and game sounds supplied 

to this plurality of players could be created by using one terminal, or they could be 

created by using a plurality of terminals (game machines or mobile phones) connected 

by a network (transmission lines or communications circuitry) or the like. 
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9. Analysis of first and second stereoscopic viewing methods 

Numerical analysis will now be performed on the first and second stereoscopic 
viewing methods of this embodiment, as described with reference to Figs. 1 and 13. The 
5 first stereoscopic viewing method shows that it is possible to reconfigure an image of a 
real-world object that cannot be projected directly (CI) onto the base surface (desk) 
screen as long as virtually accepted, by taking photographs (Figs. 3 and 4) obtained by 
camera capture (C2) and subjecting them to postprocessing (C3, the processing for 
removing perspective of Fig. 8A). Numerical analysis is performed here on the 
10 differences between this first stereoscopic viewing method and the second stereoscopic 
viewing that projects the object directly on the base surface (desk) screen. 

9.1 Numerical analysis of first stereoscopic viewing method 

First of all, a viewpoint (v), a camera screen (s), an object, and a coordinate 
15 system relating to those components are defined as shown in Fig. 22. In Fig. 22, 
projection from the viewpoint projects an object point (x, y, z) onto a point (x*, y*) on 
a screen (camera screen). 

First of all, the camera capture (C2) can be expressed by a combination of a 
rotation matrix Rx of Equation (1) and a projection matrix Pz of Equation (2), as 
20 follows: 
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In this case, the rotation matrix Rx is a matrix that causes a rotation such that the 
line-of-sight direction of the diagonal direction is parallel to the Z-axis direction. The 
projection matrix Pz is a matrix that represents the projection onto a screen (Z = s) of a 
camera from a viewpoint (Z = v). Note that a is the angle subtended between the line- 
of-sight direction and the base surface screen. 

The camera capture (C2) can therefore be expressed by Equation (3): 
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■ (3) 



where: 



x' = x/H 
y' = y/H 



This Equation (3) can be expressed as conversion equations such as Equation 



15 (4): 
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X = 



y = 



s-u 

ycosa + zsina - v 

s-u 

ycosa + zsina - u 



(ysina - zcosa ) 



(4) 



z' = s 



Assume, by way of example, that four grid points Gl = *(a, a, 0), G2 = a, 0), 
Q3 = l (_a 5 _a 3 0), and G4 = l (a, -a, 0) that form a square are set on a base surface such as 
a desk (Z = 0). Note that "t" means transform. 

These grid points Gl to G4 are projected onto grid points Gl' to G4' as shown in 
Fig. 23, by the conversions of Equations (3) and (4). The coordinates of these grid 
points Gl 1 to G4' are calculated by substituting l (a, a, 0), (-a, a, 0), l (-a, -a, 0), and *(a, -a, 
0) into *(x, y, z) of Equations (3) and (4), as shown in Equations (5) to (8) below. 
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••• (6) 
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(8) 



The postprocessing of the first stereoscopic viewing method (C3, the processing 
for removing perspective) performs projection conversion to project these grid points 
15 Gl' to G4' onto grid point Fl = t(b, b), F2 = t(-b, b), F3 = t(-b, -b), and F4 = t(b, -b) that 
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form a two-dimensional square on the photograph. In other words, this projection 
conversion projects the markers MK1 to MK4 (equivalent to Gl' to G4') of Fig. 3 onto 
the positions of markers MK1 to MK4 (equivalent to Fl to F2) of Fig. 5. 

A matrix that represents such a projection PI is obtained by solving 
5 simultaneous equations for matrix elements an, aj 2 , an,... a 33s as shown in Equation (9): 
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The conversion of the first stereoscopic viewing method, which is a 
10 combination of camera capture (C2) and postprocessing (C3), can be expressed by the 
rotation matrix Rx of Equation (1), the projection matrix Pz of Equation (2), and the 
projection matrix PI of Equation (9), enabling representation as shown by Equation 
(10): 
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where: 



x' = XIH 
y' = YIH 



..(10) 



This Equation (10) can be expressed as a conversion equation, as shown by 
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Equation (11): 



b X 
x = — x — 

^ 1 usin a 

b v - zcota "' ( 1 1 ) 



As mentioned above, the first stereoscopic viewing method can be expressed as 
numerical formula such as Equation (10) or Equation (11). 



9.2 Numerical analysis of second stereoscopic viewing method 

The conversion of the second stereoscopic viewing method of Fig. 13, in which 
the points of the object are projected directly onto a base surface screen, can be 
expressed by Equation (12), from Fig. 24. 
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where: 



x =X/H 

y* =Y I H + OCOSCC 



This Equation (12) can also be expressed by a conversion equation such as 
Equation (13): 
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1 vsina 

y-vcosa *** ( 1 3 ) 

v - + vcosa 

1 ^— 



The conversions of the second stereoscopic viewing method expressed by 
Equations (12) and (13) consist of the three conversions of parallel movement Ty 
(movement parallel to -vcosa in the y direction) of the object OB, as shown in Fig. 
25A; a projection Pz of the object OB after the parallel movement, as shown in Fig. 
25B; and parallel movement Ty (movement parallel to vcosa in the y direction) of the 
object OB after the projection, as shown in Fig. 26. 



9.3 Comparison of first and second stereoscopic viewing methods 

As described above, the conversion of the first stereoscopic viewing system can 
be expressed numerically by Equation (14) or (15) and the conversion of the second 
stereoscopic viewing method can be expressed numerically by Equation (16) or (17). 
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where: < 



x =X/H 

y = Y I H + vcosa 



(1 6) 
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5 The portions where Equations (14) and (16) differ are the items indicated by Jl 

and Kl. the portions where Equation (15) and (17) differ are the items indicated by J2 
and K2. 

These differing portions can be described intuitively with the aid of the 
drawings, as follows. In other words, the previously described second stereoscopic 

10 viewing method consists of three conversions, as shown in Figs. 25 A, 25B, and 26. The 
first stereoscopic viewing method differs from the second stereoscopic viewing method 
in the magnitude of offset in the initial parallel movement shown in Fig. 25A. In other 
words, this offset magnitude is zcota in the first stereoscopic viewing method (see Jl 
and J2 of Equations (14) and (15)). In contrast thereto, this offset magnitude in the 

15 second stereoscopic viewing method is vcosa (see Kl and K2 of Equations (16) and 
(17)). 

With this second stereoscopic viewing method, the offset magnitude (vcosa 
depends on the viewpoint (v) and the line-of-sight direction (a). With the first 
stereoscopic viewing method, on the other hand, the offset magnitude (zcota) depends 
20 on the height (z) and the line-of-sight direction (a), but not on the viewpoint (v) itself. 
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The offset magnitude (zcota) of this first stereoscopic viewing method is equal to the 
distance between a point Nl at which a perpendicular line dropped from a point (x, y 5 z) 
of the object intersects the base surface (desk) screen and a point N2 at which a line 
extending from the point (x, y, z) of the object in the line-of-sight direction, not the 
5 projection direction, intersects the base surface screen, as shown in Fig. 27. 

With this first stereoscopic viewing method the offset magnitude (zcota) of the 
parallel movement of Fig. 25A depends on the height (z). There will therefore be a 
disparity between the appearance of the stereoscopic view in the first stereoscopic 
viewing method and the appearance of the stereoscopic view in the second stereoscopic 
10 viewing method, in accordance with the height (z) of the object point (x, y, z), therein 
lies the difference between the first and second stereoscopic viewing methods. 

Note that the present invention is not limited to this embodiment and thus 
various modifications thereto are possible within the scope of the invention laid out 
herein. 

15 For example, terminology (such as graphic object/subject or polygon) that is 

derived from generic terminology defined within this document or figures (such as 

object or primitive surface) could be replaced by other terminology used within this 

document or the drawings. 

Similarly, the method of creating (generating) the image for the left eye, the 
20 image for the right eye, and the stereoscopic image is not limited to those described 

with reference to this embodiment, and various modifications thereto are possible. 

The images for stereoscopic viewing created (generated) by the methods of the 

present invention can be used in applications other than printed material for 

stereoscopic viewing or game images. 
25 The scope of the present invention also covers the generation of images for 

stereoscopic viewing by methods equivalent to the first and second methods described 

within this embodiment. 
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Part of requirements of a claim of the present invention could be omitted from a 
dependent claim which depends on that claim. Moreover, part of requirements of any 
independent claim of the present invention could be made to depend on any other 
independent claim. 

The present invention can be applied to various games (such as beat'-em-up 
games, competitive games, shooting games, robot-battling games, sports games, and 
role-playing games). 

This invention is can be applied to various image generation systems (game 
systems) such as arcade game systems, domestic game systems, large-scale attractions 
in which many players can participate, simulators, multimedia terminals, and system 
boards that create game images. 

The specification discloses the following matters about the configuration of the 
embodiments described above. 

According to one embodiment of the present invention, there is provided a 
method of fabricating printed material for stereoscopic viewing, the method 
comprising: 

creating a first image for the left eye for stereoscopic viewing; 

creating a first image for the right eye for stereoscopic viewing; 

creating a second image for the left eye by subjecting the first image for the left 
eye to correction processing for removing perspective of an image at a base surface in 
the first image for the left eye; 

creating a second image for the right eye by subjecting the first image for the 
right eye to correction processing for removing perspective of an image at the base 
surface in the first image for the right eye; and 

fabricating printed material for stereoscopic viewing based on the second image 
for the left eye and the second image for the right eye. 

In this embodiment, a second image for the left eye is created from the first 
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image for the left eye and a second image for the right eye is created from the first 
image for the right eye, by performing correction processing to remove the perspective 
of images at the base surface (such as the image of the base surface itself, images of 
objects at portions in contact with the base surface, and the like). Printed material for 
5 stereoscopic viewing is fabricated based on these second images for the left eye and the 
right eye. This makes it possible to provide printed material for stereoscopic viewing 
that enables the implementation of a more natural stereoscopic view with few 
inconsistencies in focal adjustment and depth perception. 

In the method of fabricating printed material for stereoscopic viewing, 
10 the base surface may comprise a first base surface and a second base surface 

having a predetermined angle with respect to the first base surface, and 
the method may further comprise: 

creating the second image for the left eye by performing first correction 
processing for removing perspective of an image at the first base surface in the first 

15 image for the left eye, with respect to an area of the first image for the left eye 
corresponding to the first base surface, and also performing second correction 
processing for removing perspective of an image at the second base surface in the first 
image for the left eye, with respect to an area of the first image for the left eye 
corresponding to the second base surface; and 

20 creating the second image for the right eye by performing the first correction 

processing for removing perspective of an image at the first base surface in the first 
image for the right eye, with respect to an area of the first image for the right eye 
corresponding to the first base surface, and also performing the second correction 
processing for removing perspective of an image at the second base surface in the first 

25 image for the right eye, with respect to an area of the first image for the right eye 
corresponding to the second base surface. 

Such a configuration makes it possible to cancel any unnaturalness in the depth 
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perception that might occur in a case where only one base surface is provided. Note that 
it is also possible to set three or more base surfaces. The plurality of base surfaces (first 
and second base surfaces) can be connected together. 

According to another embodiment of the present invention, there is provided a 
5 method of fabricating printed material for stereoscopic viewing, the method 
comprising: 

creating a first image for the left eye for stereoscopic viewing by photographing 
a subject and first to fourth marks for the left eye from a viewpoint position for the left 
eye, the first to fourth marks for the left eye forming a rectangle on a base surface; 
10 creating a first image for the right eye for stereoscopic viewing by 

photographing the subject and first to fourth marks for the right eye from a viewpoint 
position for the right eye, the first to fourth marks for the right eye forming a rectangle 
on the base surface; 

creating a second image for the left eye from the first image for the left eye, by 
1 5 performing correction processing which moves the first to fourth marks for the left eye 
in the first image for the left eye to positions of vertices of a rectangle; 

creating a second image for the right eye from the first image for the right eye, 
by performing correction processing which moves the first to fourth marks for the right 
eye in the first image for the right eye to positions of vertices of a rectangle; and 
20 fabricating printed material for stereoscopic viewing based on the second image 

for the left eye and the second image for the right eye. 

In this embodiment, the second image for the left eye is created by performing 
correction processing to move first to fourth marks of the first image for the left eye to 
the vertex positions of a rectangle (positions of first to fourth vertices), and, the second 
25 image for the right eye is created by performing correction processing to move the first 
to fourth marks of the first image for the right eye to the vertex positions of the 
rectangle. Printed material for stereoscopic viewing is then created based on the thus- 
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obtained second image for the left eye and second image for the right eye. The 
execution of such correction processing makes it possible to implement processing for 
removing the perspective at the base surface, for example, with simple processing. 

Note that a rectangle in this embodiment has a general meaning that also 
5 comprises "square". The rectangle formed by the first to fourth marks on the base 
surface and the rectangle formed by the vertices after the movement of the first to 
fourth marks can have equal aspect ratios. 

In the method of fabricating printed material for stereoscopic viewing, 

the base surface may comprise a first base surface and a second base surface 
1 0 having a predetermined angle with respect to the first base surface, and 

the method may comprise: 

creating the first image for the left eye for stereoscopic viewing by 
photographing the subject, the first to fourth marks for the left eye, and fifth to eighth 
marks for the left eye from the viewpoint position for the left eye, the first to fourth 

15 marks for the left eye forming a rectangle on the first base surface, the fifth to eighth 
marks for the left eye forming a rectangle on the second base surface; 

creating the first image for the right eye for stereoscopic viewing by 
photographing the subject, the first to fourth marks for the right eye, and fifth to eighth 
marks for the right eye from the viewpoint position for the right eye, the first to fourth 

20 marks for the right eye forming a rectangle on the first base surface, the fifth to eighth 
marks for the right eye forming a rectangle on the second base surface; 

creating the second image for the left eye from the first image for the left eye by 
performing first correction processing which moves the first to fourth marks for the left 
eye in the first image for the left eye to positions of vertices of a rectangle on the first 

25 base surface and also second correction processing which moves the fifth to eighth 
marks for the left eye in the first image for the left eye to positions of vertices of a 
rectangle on the second base surface, and 
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creating the second image for the right eye from the first image for the right eye 
by performing the first correction processing which moves the first to fourth marks for 
the right eye in the first image for the right eye to positions of vertices of a rectangle 
and also the second correction processing which moves the fifth to eighth marks for the 
5 right eye in the first image for the right eye to positions of vertices of a rectangle. 

Such a configuration makes it possible to cancel any unnaturalness in the depth 
perception that might occur in a case where only one base surface is provided. Note that 
it is also possible to set three or more base surfaces. In addition, the third and fourth 
marks and the sixth and fifth marks could be used in common. 
10 The method of fabricating printed material for stereoscopic viewing may further 

comprise: 

creating the first image for the left eye by photographing a subject from a 
viewpoint position for the left eye; 

creating the first image for the right eye by photographing the subject from a 
1 5 viewpoint position for the right eye; and 

when a distance between the subject and a viewpoint position has been extended, 
extending a distance between the viewpoint position for the left eye and the viewpoint 
position for the right eye in accordance with the change in length of the distance 
between the subject and the viewpoint position. 
20 Note that the viewpoint position could be a central point between the viewpoint 

position for the left eye and the viewpoint position for the right eye, by way of example. 

The method of fabricating printed material for stereoscopic viewing may further 
comprise: 

creating the first image for the left eye by photographing a subject from a 
25 viewpoint position for the left eye; 

creating the first image for the right eye by photographing the subject from a 
viewpoint position for the right eye; and 
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moving a viewpoint position along a line having a predetermined angle with 
respect to the base surface, when a distance between the subject and the viewpoint 
position is to be changed. 

In the method of fabricating printed material for stereoscopic viewing, 
5 a surface on which printed material for stereoscopic viewing is placed during 

stereoscopic viewing may be set to be the base surface. 

This makes it possible to implement an optimal stereoscopic view that has a 
feeling of reality when the printed material for stereoscopic viewing is placed parallel to 
a mounting surface, by way of example. 
10 The method of fabricating printed material for stereoscopic viewing may further 

comprise: 

fabricating printed material for stereoscopic viewing by combining the second 
image for the left eye and the second image for the right eye by anaglyph processing. 

Note that the printed material for stereoscopic viewing in accordance with this 
1 5 embodiment could also be created by a method other than anaglyph processing. 

According to a further embodiment of the present invention, there is provided 
printed material for stereoscopic viewing fabricated by one of the above methods. 
According to still another embodiment of the present invention, there is provided 
printed material for stereoscopic viewing fabricated by copying printed material for 
20 stereoscopic viewing fabricated by one of the above methods. 

According to a still further embodiment of the present invention, there is 
provided printed material for stereoscopic viewing which is fabricated by combining an 
image for the left eye and an image for the right eye, 

wherein each of the image for the left eye and the image for the right eye 
25 comprises an image of an object which is disposed on a base surface, and 

wherein the image of the object in the image for the left eye and the image of 
the object in the image for the right eye match at the base surface, and an offset between 
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the image of the object in the image for the left eye and the image of the object in the 
image for the right eye becomes larger with increasing distance from the base surface. 

In this embodiment, the object image of the image for the left eye and the object 
image of the image for the right eye match at the base surface position. The print 
5 position of the image for the left eye and the print position of the image for the right eye, 
and the display position of the image for the left eye and the display position of the 
image for the right eye match at the base surface position, by way of example. The 
offset between the object image of the image for the left eye and the object image of the 
image for the right eye (such as the offset in the print position or display position) 

10 grows larger with increasing distance from the base surface (increasing distance in a 
predetermined direction from the base surface). This makes it possible to implement a 
stereoscopic view having a natural feeling of reality, which has been difficult to obtain 
in a conventional stereoscopic view. 

According to yet another embodiment of the present invention, there is provided 

15 a method of fabricating printed material for stereoscopic viewing, the method 
comprising: 

creating an image for the left eye by rendering a projection of each point of a 
graphic object onto a base surface which is not orthogonal to a line-of-sight direction, in 
a projection direction linking a viewpoint position for the left eye to each point of the 
20 graphic object; 

creating an image for the right eye by rendering a projection of each point of the 
graphic object onto the base surface, in a projection direction linking a viewpoint 
position for the right eye to each point of the graphic object; and 

fabricating printed material for stereoscopic viewing, based on the image for the 
25 left eye and the image for the right eye. 

In this embodiment, an image for the left eye is created by rendering a 
projection of each point of the graphic object with respect to the base surface, in the 
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projection direction linking the viewpoint position for the left eye to the points of the 
graphic object. An image for the right eye is also created by rendering a projection of 
each point of the graphic object with respect to the base surface, in the projection 
direction linking the viewpoint position for the right eye to the points of the graphic 
5 object. Printed material for stereoscopic viewing is fabricated based on these images for 
the left eye and the right eye. This makes it possible to provide printed material for 
stereoscopic viewing that enables the implementation of a more natural stereoscopic 
view with few inconsistencies in focal adjustment and depth perception. Note that the 
base surface is not orthogonal to the line-of-sight direction (the direction linking the 

1 0 central point between the viewpoint position for the left eye and the viewpoint position 
for the right eye with the point of regard of the virtual camera), by way of example. In 
other words, the base surface differs from a transparency conversion screen that is 
orthogonal to the line-of-sight direction. 

In the method of fabricating printed material for stereoscopic viewing, the 

15 graphic object which is projected onto the base surface may be a graphic object which 
is disposed on the base surface. In the method of fabricating printed material for 
stereoscopic viewing, the graphic object which is projected onto the base surface may 
be a graphic object at least part of which is disposed at a further side from the base 
surface as seen from the viewpoint position for the left eye and the viewpoint position 

20 for the right eye. 

In the method of fabricating printed material for stereoscopic viewing, 
the base surface may comprise a first base surface and a second base surface 
having a predetermined angle with respect to the first base surface, and 
the method may comprise: 

25 creating an image for the left eye by rendering a projection of each point of a 

graphic object onto a first or second base surface, in a projection direction linking a 
viewpoint position for the left eye to each point of the graphic object, onto the first or 
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second base surface; and 

creating an image for the right eye by rendering a projection of each point of the 
graphic object onto the first or second base surface, in a projection direction linking a 
viewpoint position for the right eye to each point of the graphic object, onto the first or 
5 second base surface. 

Such a configuration makes it possible to cancel any unnaturalness in the depth 
perception that might occur in a case where only one base surface is provided. Note that 
it is also possible to set three or more base surfaces. The plurality of base surfaces (first 
and second base surfaces) can be connected together. 
1 o The method of fabricating printed material for stereoscopic viewing may further 

comprise: 

when a distance between the graphic object and a viewpoint position has been 
extended, extending a distance between the viewpoint position for the left eye and the 
viewpoint position for the right eye in accordance with the change in length of the 
1 5 distance between the graphic object and the viewpoint position. 

The method of fabricating printed material for stereoscopic viewing may further 
comprise: 

moving a viewpoint position along a line having a predetermined angle with 
respect to the base surface, when a distance between the graphic object and the 
20 viewpoint position is to be changed. 
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